The development of a web-based monitoring system for evaluating student learning
Dennis C S Law and Patrick P C Yip
Caritas Francis Hsu College
Hong Kong SAR, China
This paper describes the plan for developing a Student Learning Evaluation on the Web (SLEW) system which aims to apply web-based technology to collect student feedback on their learning. When fully developed, the SLEW system will serve as a monitoring system for analysing student data gathered on a regular (e.g. annual) basis, especially students’ learning patterns (in terms of their processing strategies, regulation strategies, learning orientations and conceptions of learning) and their perceptions of their learning contexts. The monitoring system will be an important source of profiling and diagnostic data on student learning for the quality assurance (QA) activities of post-secondary institutions, e.g.
Profiling data on students’ perceptions of their learning contexts in a cross-sectional manner can assist the institutions to identify possible areas for improvement in their teaching and learning arrangements.
Profiling data on students’ learning patterns longitudinally can help the institutions to track their students’ development in learning (e.g. the appropriate development of regulation strategies in students).
Diagnostic data on student learning gathered on an individual basis can assist the promotion of students’ reflection on their own learning and can also provide student advisers with objective evidence for use in counselling sessions with students on their learning, especially with the group of at-risk students whose learning problems may have arisen in their secondary school studies.
Currently, the groundwork for the SLEW system has been established through the selection and adaptation of two research instruments developed and validated in Western higher education contexts: the Course Experience Questionnaire (CEQ) and the Inventory of Learning Styles (ILS). The adapted instruments were tested on a research platform comprising six Schools in the Caritas Community and Higher Education Service (which has about 2,500 students). The appropriateness of these instruments for application in a Hong Kong setting was verified mainly through calculating Cronbach alphas to assess the internal consistency of the instruments’ scales and conducting exploratory factor analyses to assess the construct validity of the scales in relation to empirical structure. Through multiple regression analyses, the systematic relationships between the relevant student learning constructs operationalized by the CEQ and the ILS were also explored.
Although some cultural effects were identified, the findings indicated that, to a large extent, the CEQ and the ILS can capture the variation in students’ perceptions of their learning contexts and their learning patterns in Hong Kong post-secondary education, thus paving the way for the development of the SLEW system.
At present, the targeted platform for the SLEW system comprises two post-secondary institutions in Hong Kong. With the implementation of SLEW, the project team aims to change the quality culture of these two institutions by introducing transformation into their internal QA practices, particularly by:
applying web-based technology appropriately in the institutions’ QA activities (instead of using conventional methods);
administering instruments that are developed and validated rigorously to collect credible data on student learning to inform the institutions’ QA activities (instead of basing them on data-collection instruments developed in an ad hoc manner); and
focusing the accountability aspects of the institutions’ QA activities (which are based largely on institution-level data) more on the improvement aspects (which can be based largely on student-level data collected via the SLEW system), so as to strike a good balance.
If the SLEW system is implemented successfully through the present project, the project team sees its potential for influencing the quality culture within Hong Kong’s postsecondary education sector by sharing good practices with other institutions.